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SUMMARY 


This  final  report  describes  research  on  stochastic  and  adaptive 
systems  by  faculty  and  students  of  the  Decision  and  Control  Sciences 
Group  of  the  M.I.T.  Laboratory  for  Information  and  Decision  Systems 
(formerly  Electronic  Systems  Laboratory)  with  support  provided  by  the 
United  States  Air  Force  Office  of  Scientific  Research  under  Grant 
AFOSR  77-3281B.  The  Grant  Monitor  was  Charles  L.  Nefzger,  Major,  USAF. 

The  time  period  covered  by  this  report  is  February  1,  1978  to  January  31, 

1979- 

Substantial  progress  is  reported  in  the  areas  of  nonlinear  filtering, 
parameter-adaptive  control,  reliable  control  system  design,  and  reduced 
order  compensators  for  singular  stochastic  control  problems. 


1 . INTRODUCTION 


During  the  grant  period  of  February  1978  - January  31,  1979  of 
AFOSR  Grant  77-3281B,  work  has  progressed  on  the  following  main  subject 
areas: 

(i)  Non-linear  Filtering 

(ii)  Stochastic  Control 

(iii)  Adaptive  Control 

(iv)  Reliable  and  Robust  Control  System  Designs 

(v)  Singular  Stochastic  Control  and  Dual  .onpensator  Structures 

(vi)  Nonlinear  Control  Law  Synthesis 

(vii)  Control  Laws  Implemented  by  Finite-state  Sequential  Machines. 
The  work  was  carried  out  under  the  joint  direction  of  Professors  M. 

Athons  and  S.K.  Mitter.  They  were  assisted  by  Professor  T.  Johnson,  Dr. 
Mark  Davis  (visiting  from  Imperial  College  and  supported  by  the  grant). 
Dr,  R.  Singh  (visitor),  Mr,  D.  Birdwell  (fellowship  student),  Mr.  R.  Ku, 
Mr.  D.  Oconc  (fellowship  student),  Mr,  P.  Parikh  and  Mr.  D.  Wimpey. 
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2.  NONLINEAR  FILTERING 

During  this  grant  period  Professor  Mitter  has  continued  his  work  on 
non-linear  filtering.  In  this  work  he  was  assisted  by  Dr.  Mark  Davis 
(visiting  from  Imperial  College  and  supported  by  the  current  grant)  and 
fir.  D.  Ocone  (fellowship  student). 

We  have  concentrated  on  the  following  aspects  of  the  non-linear 
filtering  problems: 

(i)  Robust  Non-linear  Filtering 

(ii)  Filtering  for  Systems  with  Multiplicative  Noise 

(iii)  Rounds  on  Non-linear  Filtering 

(iv)  Perturbation  Theory  for  Non-linear  Filtering. 

Robust  Non-linear  Filtering 

During  the  last  few  years  there  has  been  intense  activity  in  the 
modelling  of  stochastic  dynamical  systems.  A systematic  approach  to  these 
questions  was  given  by  McShane  [ 1 ] . 

Subsequently  Balakrishnan  (see  [2]  and  references  cited  therein),  and 
Sussman  [3]  have  explored  different  aspects  of  this  problem.  The  basic  ques- 
tion has  to  do  with  the  fact  that  modelling  of  noise  processes  as  Ito-processes 
is  apparently  not  the  right  thing  to  do  in  the  implementation  of  filters.  The 
reason  is  that  physical  noise  processes  have  finite  bandwidth  and  the  formula 
for  computing  conditional  expectations  using  the  Ito-model  (infinite  bandwidth 
process)  when  applied  to  physical  processes  will  not  give  the  correct  answer. 
Balakrishnan  and  Mitter  [U]  have  argued  that  a white  noise  model  as  proposed 
by  Segal  and  Gross  is  much  more  appropriate  when  non-linear  filters  are  to 
be  used  in  real  physical  situations.  Work  on  this  is  progressing.  Using 
certain  ideas  of  Clark  we  are  able  to  give  a robust  version  of  the  likelihood- 
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ratio  formula  where  the  filter  turns  out  to  be  a smooth  function  of  the 
physical  data.  These  ideas  are  also  related  to  the  innovations  problem. 

Here  we  have  been  able  to  obtain  positive  results  for  a significant  class 
of  non-linear  filtering  problems. 

Filtering  for  Systems  with  Multiplicative  Noise 

The  goal  of  this  research  is  to  determine  explicitly  the  non-linear 
filter  for  state  and  observation  processes  which  are  linear  but  where  the 
noise  enters  multiplicatively.  This  class  of  models  turns  out  to  be  quite 
general  since  the  statistical  description  of  some  core  general  non-linear 
models  can  be  approximated  arbitrarily  closely  by  models  of  this  type. 

Closed  form  solutions  for  a special  class  of  multipxicative  noise  models 
have  been  obtained  [5].  Work  is  now  progressing  in  trying  to  determine 
the  best  polynomial  filter  for  the  more  general  class  of  linear  systems 
with  multiplicative  noise.  We  would  like  to  mention  that  the  class  of 
systems  we  are  considering  is  probably  the  only  one  for  which  the  best 
linear  filter  can  be  computed  as  a finite  dimensional  filter. 

Bounds  on  Non-linear  Filtering 

In  the  doctoral  thesis  of  Galdos, written  under  the  direction  of 
Professor  Mitter  and  supported  by  a previous  grant  of  the  AFOSR,  a formula 
for  lower  bound  of  the  filtering  error  was  given.  This  is  an  a-priori  bound 
which  was  obtained  using  information-theoretic  ideas.  The  basic  problem  with 
this  bound  is  that  it  is  very  difficult  to  evaluate.  Work  is  now  progressing 
to  see  if  the  bound  can  be  evaluated  in  the  asymptotic  sense.  It  appears 
that  the  evaluation  of  the  asymptotic  error  can  be  solved  in  certain  situations 
by  solving  an  eigenvalue  problem  for  a certain  elliptic  partial  differential 
operator.  This  work  will  be  reported  later. 
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Perturbation  Theory  for  Non- linear  Filterin, 

In  recent  work  [It]  Professor  Mitter  has  shown  that  the  conditional 
density  equations  for  the  Kalman  filtering  problem  has  a precise  analogy 
to  the  free  quantum  field.  This  correspondence  sheds  new  light  even  on 
the  Kalman  Filtering  Problem  and  indicates  how  a probabilistic  interpre- 
tation can  be  given  to  the  fast  algorithms  of  Kailath  and  others  for  the 
Kalman  filter. 

This  has  suggested  how  perturbation  theory  can  be  done  systematically 
for  non-linear  filtering  problems.  In  particular  i!r.  D.  Ocone  (in  a forth- 
coming Ph.D.  thesis)  has  shown  how  convergent  Volxerra  expansions  can  be 
obtained  to  express  conditional  expectations  of  functions  of  the  state  process. 


REFERENCES  (Section  2) 


1.  E.J.  McShane:  Stochastic  Calculus  and  Stochastic  Models,  Academic 
Press,  New  York,  1974. 

2.  A.V.  Balakrishnan:  "Likelihood  Ratios  for  Signals  in  White  Noise", 
Applied  Mathematics  and  Optimization,  1977- 

3.  H.J.  Sussmann:  "On  the  Gap  between  Deterministic  and  Stochastic  Ordinary 
Differential  Equations",  Ann,  of  Prob.,  (S,  pp.  19-4l,  1978. 

1*.  S.K.  Mitter:  "Modelling  for  Stochastic  Systems  and  Quantum  Fields", 
Proceedings  of  the  IEEE  Decision  and  Control  Conference.  San  Diego, 
California,  January  1979. 

5.  S.  Marcus,  S.K.  Mitter  and  D.  Ocone:  "Finite  Dimensional  Estimation 
for  a Class  of  Non-linear  Systems" , Proceedings  of  the  International 
Conference  on  Stochastic  Systems.  Oxford,  England,  1978  (to  be  published 
by  Academic  Press). 
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3.  ADAPTIVE  CONTROL 

During  the  past  year,  the  completed  Ph.D.  thesis  of  R.  Ku  [l]  under 
the  supervision  of  Professor  Athans  has  demonstrated  that  even  in  the  case 
of  scalar  linear  systems,  if  both  the  state  equation  and  the  measurement 
equation  contain  white  parameters,  the  resulting  optimal  control  problem, 
with  respect  to  quadratic  performance  criteria,  admits  no  closed  form  analy- 
tical solution.  Rather  it  leads  to  a complex  two-point  boundary  value  prob- 
lem. Numerical  methods  have  been  developed  to  solve  this  nonlinear  two- 
point-boundary  value  problem  which  demonstrates  that  the  uncertainty  threshold 
principle  still  exists,  and  that  the  threshold  for  the  existence  of  infinite- 
horizon  optimal  decision  rules  is  decreased  when  noisy  measurements  of  the 
state  are  made  under  both  multiplicative  and  additive  white  noise.  It  should 
be  self-evident  that  the  lack  of  analytic  solutions  to  the  problem  will  be 
present  in  the  multivariable  case. 

In  spite  of  these  analytical  difficulties  it  was  decided  to  study 
further  the  multivariable  version  of  the  problem.  A study  was  initiated 
by  Mr.  P.  Parikh  and  Professor  Athans  in  which  the  stochastic  optimal  solution 
was  derived  for  Linear-Quadratic  regulator  and  tracking  problems  under  the 
assumption  of  exact  state  measurements.  General  purpose  computer  subroutines 
have  been  developed  by  Mr.  Parikh  and  Mr.  Carrig  under  the  supervision  of 
Professor  Athans,  and  extensive  digital  computer  simulation  of  a simple  two 
state  two  input  discrete  time  example  have  been  carried  out. 

Simultaneously  with  this  effort,  we  have  developed  a set  of  sensitivity 
equations  for  the  stochastic  control  problem.  The  basic  idea  is  that  in  a 
stochastic  optimal  control  problem  the  impact  of  the  variability  of  a certain 
system  parameter  (or  sets  of  parameters)  can  be  only  assessed  through  the  use 
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of  a specific  perfonnance  indec.  Thus,  we  are  investigating  the  effect 
of  the  variance  of  a given  parameter  \ipon  the  optimal  cost-to-go.  Mote 
that  for  certain  objective  functions  the  variance  of  a given  parameter  may 
be  dominant,  while  for  other  objective  functions  the  variance  of  this  un- 
certain parameter  may  not  be  as  significant.  Simulation  results  are  showing 
this  effect  quite  clearly.  If  the  uncertain  parameters  are  characterized  by 
small  standard  deviations,  the  response  of  the  system  is  relatively  insensi- 
tive to  the  selection  of  the  weighting  matrices  in  a tracking  linear-quadratic 
context.  However,  as  the  parameter  variances  increase  one  sees  significant 
deviations  in  the  time  trajectories  of  the  control  and  state  variables,  thus 
illustrating  the  point  that  the  effects  of  parameter  uncertainty  cannot  be 
decoupled  from  the  overall  system  objective.  The  first  documentation  of  these 
results  will  be  in  the  3.M.  thesis  of  P.  Parikh  scheduled  for  completion  in 
April  1979- 

Since  September  1978  Professor  Athans  and  Professor  Kendrick  (visiting 
from  the  University  of  Texas)  have  been  collaborating  in  the  area  of  adaptive 
control  so  as  to  understand  better  the  nature  of  the  probing  terms  and  of  the 
caution  terms  in  the  dual  control  adaptive  algorithm.  Extensive  simulations 
have  been  carried  out  and  more  are  necessary  to  understand  fully  under  what 
circumstances  active  learning  adaptive  control  leads  to  a superior  performance 
as  compared  to  passive  learning  using  (a)  the  certainty  equivalence  method  and 
(b)  the  open  loop  feedback  optimal  method.  A simple  two  state  discrete-time 
example,  representing  a highly  aggregated  model  of  the  U.S.  economy  is  used  for 
simulation  purposes.  This  research  is  scheduled  for  completion  in  May  1979* 

Professor  Athans,  Professor  Kendrick,  and  Mr.  Dersin  are  also  collaborat- 
ing in  the  preparation  of  a paper  that  also  deals  with  the  dual  control  algorithm. 
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and  in  particular  with  improved  definitions  of  the  probing  and  caution 
terms  and  with  the  relationship  of  the  du^l  method  equations  to  those 
obtained  under  the  assumption  that  the  system  parameters  are  white 
(uncorrelated  in  time).  A paper  documenting  the  results  of  this  study 
will  become  available  in  May  1979- 
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k . RELIABLE  AND  ROBUST  CONTROL  SYSTEM  DESIGNS 

Mr.  J.D.  Birdwell,  Professor  M.  Athans,  and  Dr.  D.A.  Castanon  have  con- 
tinued during  the  past  year  their  investigations  in  the  area  of  stochastic 
control  with  special  emphasis  on  developing  a method  of  approach  and  theoret- 
ical framework  which  advances  the  state  of  the  art  in  the  design  of  reliable 
multivariable  control  systems,  with  special  focus  on  actuator  failures  and 
necessary  actuator  redundancy  levels. 

The  mathematical  model  consists  of  a linear  tine  invariant  discrete 
time  dynamical  system.  Configuration  changes  in  the  system  dynamics,  (such 
as  actuator  failures,  repairs,  introduction  of  a back-up  actuator)  are  governed 
by  a Markov  chain  that  includes  transition  probabilities  from  one  configuration 
state  to  another.  The  performance  index  is  a standard  quadratic  cost  function- 
al, over  an  infinite  time  interval. 

If  the  dynamic  system  contains  either  process  white  noise  and/or  noisy 
measurements  of  the  state,  then  the  stochastic  optimal  control  problem  reduces, 
in  general,  to  a dual  problem,  and  no  analytical  or  efficient  algorithmic  solu- 
tion is  possible.  Thus,  the  results  are  obtained  under  the  assumption  of  full 
state  variable  measurements,  and  in  the  absence  of  additive  process  white  noise. 

Under  the  above  assumptions,  the  optimal  stochastic  control  solution  can 
be  obtained.  The  actual,  system  configuration,  i.e.  failure  condition,  can  be 
deduced  with  an  one-step  delay.  The  calculation  of  the  optimal  control  law 
requires  the  solution  of  a set  of  highly  coupled  Riccati-like  matrix  difference 
equations;  if  these  converge  (as  the  terminal  time  goes  to  infinity)  one  has 
a reliable  design  with  switching  feedback  gains,  and,  if  they  diverge*,  the 
design  is  unreliabl e and  the  system  cannot  be  stabilized  unless  more  reliable 
actuators  or  more  redundant,  actuators  are  employed.  For  reliable  designs,  the 


*(This  concept  is  similar  to  that  described  as  the  Uncertainty  Threshold  Principle. ) 


feedback  system  requires  a switching  gain  solution;  that  is,  whenever  a system 
change  is  detected,  the  feedback  gains  must  be  reconfigured.  On  the  other 
hand,  the  necessary  reconfiguration  gains  can  be  precomputed,  from  the  off-line 
solutions  of  the  Riccati-like  matrix  difference  equations. 

Through  the  use  of  the  matrix  discrete  minimum  principle,  a suboptimal 
solution  was  also  obtained.  In  this  approach,  one  wishes  to  know  whether  or 
not  it  is  possible  to  stabilize  the  system  with  a constant  feedback  gain,  which 
does  not  change  even  if  the  system  changes.  Once  more  this  can  be  deduced  from 
another  set  of  coupled  Riccati-like  matrix  diffe?'nce  equations.  If  they  di- 
verge as  the  terminal  time  goes  to  infinity,  then  a constant  gain  implementa- 
tion is  unreliable,  because  it  cannot  stabilize  the  system.  If,  on  the  other 
hand,  there  exists  an  asymptotic  solution  to  this  set  of  Riccati-like  equations 
then  a reliable  control  system  without  feedback  reconfiguration  can  be  obtained. 
The  implementation  requires  constant  gain  state  variable  feedback,  and  the  feed- 
back gain  can  be  calculated  off-line. 

In  summary,  these  results  can  be  used  for  off-line  studies  relating  the 
open  loop  dynamics,  required  performance,  actuator  mean  time  to  failure,  and 
functional  or  identical  actuator  redundancy,  with  and  without  feedback  gain 
reconfiguration  strategies. 

Documentation  of  these  results  can  be  found  in  Birdwell's  thesis  [2]  and 
a paper  [3] . 

Robustness  Re  -.earch 

Additional  results  in  the  robustness  area  have  not  been  obtained  as  yet. 

Our  plans  are  to  concentrate  upon  the  effect  of  modelling  errors  in  t.he  robust- 
ness area,  through  the  use  of  stochastic  singular  perturbation  theory. 
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References  [li]  and  [5]  related  to  robustness  have  appeared  in  the 
literature  during  this  reporting  period.  Their  contents  have  been  discussed 
in  previous  interim  reports. 
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5.  SINGULAR  STOCHASTIC  CONTROL  AND  DUAL  COMPENSATOR  STRUCTURES 

The  main  outlines  of  a theory  of  reduced-order  compensator  design 
are  now  in  place.  As  previously  described,  we  consider  the  case  of  a 
linear  time-invariant  system  subject  to  additive  Gaussian  white  noise  dis- 
turbances and  measurement  errors;  the  problem  is  to  design  a time-invariant 
compensator  of  specified  order  which  minimizes  asymptotically  the  sum  of 
variances  of  a finite  number  of  linear  combinations  of  state  and  control 
variables,  assuming  that  a set  of  stabilizing  gains  exists.  A key  to  under- 
standing this  problem  is  the  dual  singular  cases  arising  when  there  is  zero 
plant  noise  or  observation  noise,  as  indicated  in  [8]. 

In  [7],  the  conditions  for  zero-order  compensation,  or  output-feedback 
are  given.  Together,  these  special  cases  suggest  the  solution  of  the  general 
problem:  the  optimal  gains  are  computed  by  determining  two  aggregations  of 
the  plant  dynamics:  a "control  aggregation"  and  a "filtering  aggregation". 
These  two  problems  are  coupled,  but  in  a tractable  way.  Together,  the  solu- 
tions of  these  two  problems  determine  the  optimal  reduced-order  compensator 
gains.  These  results  are  currently  being  tested  by  simulation;  a detailed 
technical  report  will  follow. 
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6.  NONLINEAR  CONTROL.  LAW  GYMTHEf.TS 

Dr.  R.N.P.  Singh,  a Visiting  Scientist  at  M.I.T.,  and  Professor  Johnson 
have  studied  the  problem  of  developing  functional  expansion  techniques  for 
the  solution  of  the  Hamilton-Jacobi-Bellman  equation  which  characterizes 
optimal  nonlinear  feedback  laws  ([2],  [!»]).  Functional  expansion  techniques 
differ  from  series  expansion  techniques  in  that  at  each  iteration  the  entire 
functional  form  of  the  solution  may  be  modified;  thus  there  is  a potential 
for  faster  convergence  of  numerical  methods  of  this  type  (this  was  recognized 
by  Bellman  many  years  ago).  We  have  looked  at  a certain  class  of  nonlinear 
problems  where  advantage  is  to  be  gained  by  taking  a quadratic  initial  guess 
for  the  optimal  cost-to-go;  this  has  the  advantage  of  simplifying  later  terms 

I 

in  the  series  expansion,  facilitating  a recursive  computation  of  the  optimal 

: ; |j 

feedback  law.  We  are  continuing  to  investiage  the  possibility  of  further 

analytical  results  which  may  be  obtained  using  the  method  of  characteristics. 

r 

i I 


7-  CONTROL  LAWS  IMPLEMENTED  BY  FINITE-STATE  SEQUENTIAL  MACHINES 


This  avenue  of  research  has  proved  so  promising  that  a separate  prop- 
osal has  "been  submitted  for  its  continuation.  During  the  period  of  the 
current  grant,  we  have  investigated  mappings  from  continuous  to  discrete 
variables  as  well  as  certain  methods  for  the  analysis  of  feedback  systems 
containing  continuous  and  discrete  parts.* 

The  representation  of  mappings  between  continuous  domain  and  discrete 
range  is  considered  to  be  a crucial  issue  in  the  design  and  analysis  of  hybrid 
systems.  A knowledge  of  the  general  structure  ana  properties  of  such  mappings 
could  lead  to  a synthesis  procedure  for  synchronous  encoders  and  decoders.  In 
the  sampled-data  case,  we  have  investigated  the  notion  of  an  "acceptor"  [l], 
which  may  be  viewed  as  a continuous/discrete  mapping  realizable  by  a continuous 
finite-dimensional  system  followed  by  a threshold  device.  The  function  of  an 
acceptor  is  to  distinguish  one  class  of  input  sequences  from  another.  A '/ery 
broad  class  of  (single-input,  single-output)  acceptors  admits  a shift  realiza- 
tion, wherein  the  classification  is  decided  as  a discontinuous  memoryless  func- 
tion of  the  stored  past  input  value.  The  advantage  of  the  acceptor  formulation 
is  that  the  action  of  the  device  can  be  specified  in  advance;  it  is  a generali- 
zation of  the  notion  of  acceptor  from  automata  theory.  We  have  also  considered 
the  possibilities  of  designing  continuous-time  encoders;  in  [6]  it  is  shown 
that  one  can  devise  an  asynchronous  coder  which  is  continuous.  This  suggests 
that  asynchronous  coding  may  preserve  information  content,  whereas  in  general 
synchronous  coding  cannot. 

Several  initial  ideas  have  been  developed  in  the  area  of  control  of  hybrid 
systems.  We  have  extended  the  results  of  [3]  to  show  that,  at  least  in  some 


* We  refer  to  continuity  of  the  state-transition  map  with  respect  to  the 
state  variables,  not  with  respect  to  time. 
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cases, a closed-loop  hybrid  system  admits  a state-space  realization  in  terms 
of  transition  times,  continuous  system  states  and  discrete  system  states. 

With  this  realization,  the  state  may  be  propagated  directly  from  one  transi- 
tion time  to  the  next,  without  the  need  for  "simulation"  of  the  continuous 
system.  This  realization  also  bears  a closer  resemblance  to  the  models  used 
in  queueing  and  jump  process  theories.  In  the  sampled-data  case,  D.N.  Wimpey 
has  pursued  the  possibility  of  designing  finite-state  compensators  by  means 
of  exact  finite-state  descriptions  the  input-output  properties  of  a continuous 
plant  viewed  through  a discontinuous  input  and  output  device.  Existing  results 
from  automata  theory  may  then  be  applied  to  design  a finite-state  regulator 
for  a finite-state  system.  A report  is  in  preparation.  S.N.  Jones  [5]  has 
proposed  a procedure  for  design  of  stochastic  finite-state  controllers  for  un- 
certain discontinuous  systems. 
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